Totagraphy: Coherent Diffractive/Digital Information Reconstruction by Iterative Phase Recovery Using Special Masks

ABSTRACT

A totagram is produced by an iterative spectral phase recovery process resulting in complete information recovery using special masks, without a reference beam. Using these special masking systems reduce computation time, number of masks, and number of iterations. The special masking system is (1) a unity mask together with one or more bipolar binary masks with elements equal to 1 and −1, or (2) a unity mask together with one or more phase masks, or (3) a unity mask together with one pair of masks or more than one pair of masks having binary amplitudes of 0&#39;s and 1&#39;s, in which the masks in the pair are complementary to each other with respect to amplitude, or (4) one or more pairs of complementary masks with binary amplitudes of 0&#39;s and 1&#39;s without a unity mask.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a divisional of U.S. patent application Ser. No.17/120,919, filed Dec. 14, 2020 and issued as U.S. Pat. No. ______,which is hereby incorporated herein by reference in its entirety.

TECHNICAL FIELD

The present invention relates to phase recovery systems and methods. Inparticular, amplitude and phase are reconstituted for a coherent waveafter measuring its amplitude at a spectral output.

BACKGROUND ART

Information embedded in terms of amplitude and phase as in coherent waverepresentations leads to applications which are 1 or higher dimensionalas in imaging. In such systems, phase is often more important thanamplitude. In many coherent systems, phase is lost because what ismeasurable is intensity which is proportional to the square of theamplitude. Phase might also be intentionally lost. Phase recovery isalso important with one-dimensional signals in a number of applicationssuch as speech recognition, blind channel estimation, and blinddeconvolution. The phase problem goes back to Rayleigh who wrote aboutit in 1892. Phase recovery has been a celebrated problem in succeedingyears, and this process has accelerated after the 1960's when the laserand other important sources of coherent radiation were discovered.

There are indirect ways to recover phase and thereby achieve completeinformation recovery, for example, in diffractive imaging resulting in3-D information. Holography discovered by Dennis Gabor is one of them,and it achieves 3-D imaging by introducing a reference wave. This has alot to do with modulation principles used in communications. Another wayis closely related to the Gerchberg-Saxton algorithm (1971-72), alsoknown as the original Gerchberg-Saxton algorithm, and referred to as“GSA” herein, which involves measurements on two related planes, theinput plane and the output spectral plane. Advances in a number of areasin science and technology are related to the GSA published in 1972 [R.W. Gerchberg, W. O. Saxton, “A practical algorithm for the determinationof the phase from image and diffraction plane pictures,” Optik, Vol. 35,pp. 237-246, 1972].

R. W. Gerchberg later made an improvement to the GSA by introducing Nindependent measurement systems on two planes especially by using phasemasks. This improvement is known herein as “Gerchberg's second method,”or “G2.” G2 was published in R. W. Gerchberg, “A New Approach to PhaseRetrieval of a Wave Front,” Journal of Modern Optics, 49:7, 1185-1196,2002, incorporated by reference in its entirety herein. Further aspectsof G2 are described in U.S. Pat. Nos. 6,369,932 B1; 6,545,790; and8,040,595 all incorporated by reference in their entirety herein.

Unlike holography, G2 does not require a reference wave. Rather, G2 issimilar to measuring a quantity of interest in N independent ways andthen doing averaging between the results. The patents show how toachieve this in practice when using waves. G2 is believed to be thefirst such method using multiple measurements for reliable phaserecovery. Some other well-known methods for phase recovery are the errorreduction (ER) algorithm [, J, R, Fienup, ‘Reconstruction of an objectfrom its Fourier transform,’ Optics Letters, Vol. 3, No 1, pp. 27-29,July 1978; J. R. Fienup, ‘Phase retrieval algorithms, a comparison,’Applied Optics, Vol. 21, No. 15, pp. 2758-2769, 1 Aug. 1982], theaveraged successive relaxations (ASR) [J. C. H. Spence, ‘Diffractive(lensless) imaging,’ Ch. 19, Science of Microscopy, edited by P. W.Hawkes, J. C. H. Spence, Springer, 2007], the hybrid projectionreflections (HPR) [H. H. Bauschke, P. L. Combettes, D. Russell Luke,‘Hybrid projection-reflection method for phase retrieval,’ J. OpticalSoc. Am. A, Vol. 20, No. 6, pp. 1025-1034, June 2003], relaxed averagedalternating reflections (RAAR) [D. Russell Luke, ‘Relaxed averagedalternating reflections for diffraction imaging,’ Inverse Problems, Vol.21, pp. 37-50, 2005], oversampling smoothness (OSS) [J. A. Rodriguez, R.Xu, C.-C. Chen, Y. Zou, and J. Miao, ‘Oversampling smoothness: aneffective algorithm for phase retrieval of noisy diffractionintensities,’ J. Applied Crystallography, Vol. 46, pp. 312-318, 2013]and difference maps (DM) [V. Elser, ‘Solution of the crystallographicphase problem by iterated projections,’ Acta Crystallography. Section A:Foundations Crystallography, Vol. 59, pp. 201-209, 2003]. There are anumber of algorithms considerably more recent, utilizing more effectiveoptimization methods such as SO2D and SO4D [Stefano Marchesini, ‘Phaseretrieval and saddle-point optimization,’ J. Optical Soc. Am. A, Vol.24, No. 10, pp. 3289-3296, October 2007]. A new benchmark study of manypopular phase retrieval algorithms is discussed in PhasePack [R.Chandra, T. Goldstein, C. Studer, ‘Phasepack: a phase retrievallibrary,’ IEEE 13^(th) international conference on sampling theory andapplications, pp. 1-5, 2019]. In this work, averaging and masking with 8bipolar binary masks at the system input is used with 12 iterative phaserecovery methods in the same way as in G2.

The common theme in all these algorithms is to achieve best phaserecovery by using prior information and constraints. Use of input masksleads to such prior information. Nonnegativity, support information, andamplitude information are also commonly used as prior information.Support information is especially important. This often means the(complex) image of size N×N is at the center of a window surrounded byzeros to make the total size 2N×2N. This is also important when usingthe fast Fourier transform (FFT) to approximate the continuous Fouriertransform in digital implementations.

Experimental work indicates that there is usually not enough priorinformation with a single measurement of amplitudes in the Fourierdomain for perfect phase and image recovery. In other words, therecovery results with given data may be better with some methods thanothers, but the recovery is usually not perfect, namely it is oftenapproximate without additional information. Works involving multiplemeasurements by using input masks outlined above make up for thisdeficiency.

Recently, machine learning, and especially deep learning methods havebeen utilized, often for improving the results obtained with previousphase recovery methods. For example, two deep neural networks (DNNs)have been used together with the HIO method to improve the phaserecovery results [Ç. Işll, F. S. Oktem, and A. Koç, ‘Deep iterativereconstruction for phase retrieval,’ Applied Optics, Vol. 58, pp.5422-5431, 2019].

First, a DNN is used iteratively with the HIO method to improve thereconstructions. Next, a second DNN is trained to remove the remainingartifacts.

There is a growing realization in the research community that multiplemeasurements are necessary if high quality phase and image recovery arerequired. Quite recently, a number of such methods have been publishedin the literature. Below a discussion is presented on some methodshaving multiple measurements with some similarity to Gerchberg's G2method.

In the phaselift method by Candes et al. [E. J. Candes, Y. Eldar, T.Strohmer, V, Voroninski, ‘Phase Retrieval via Matrix Completion,’preprint, August 2011; E. J. Candes, X. Li, M. Soltanolkotabi, ‘PhaseRetrieval from Coded Diffraction Patterns,’ Stanford University,Technical Report No. 2013-12, December 2013] the initial approach is thesame as in Gerchberg's G2 method. In other words, a number ofmeasurements are taken by using a number of masks. They also mention theuse of optical grating, ptychography and oblique illuminations assubstitutes for masks. However, masks are the major mechanism used intheir papers. The averaging step in G2 is replaced by a convexoptimization method, which is also related to the matrix completion ormatrix recovery problems.

In the Fourier-weighted projections method by Sicairos and Fienup [M.Guizar-Sicairos, J. R. Fienup, ‘Phase Retrieval with Fourier-WeightedProjections,’ J. Optical Soc. Am. A, Vol. 25, No. 3, pp. 701-709, March2008], masks are also used to achieve high quality phase recovery. Theypropose different types of masks for this purpose.

Ptychography is another method which utilizes multiple diffractionintensity measurements [J. M. Rodenburg, ‘Ptychography and RelatedImaging Methods,’ Advances in Imaging and Electron Physics, Vol. 150,pp. 87-184, 2008]. It was first introduced by Hoppe in the time period1968-1973, especially for X-ray imaging. Ptychography relies onrecording at least 2 diffraction intensities by shifting theillumination function or the aperture function with respect to theobject to be imaged by a known amount instead of relying on masks. Thus,there is a moving probe which illuminates the object at a time. Whenthere is sufficient amount of overlap between the different parts ofillumination, phase recovery can be achieved by an iterative phaseretrieval algorithm. Another related algorithm has recently beendeveloped by Sicairos and Fienup based on diverse far field intensitymeasurements taken after translating the object relative to the knownillumination pattern [M. Guizar-Sicairos, J. R. Fienup, ‘Phase Retrievalwith Transverse Translation Diversity: A Nonlinear OptimizationApproach,’ Optics Express, Vol. 16, No. 10, pp. 7264-7278, 12 May 2008].In this work, nonlinear optimization is used.

In summary, multiple diffraction intensity measurements are currently inuse in the research community to solve phase and image recoveryproblems, for example, leading to diffractive (lensless) imaging [B.Abbey et al, ‘Lensless Imaging Using Broadband X-Ray Sources,’ NaturePhotonics, pp. 420-424, 26 Jun. 2011]. This is especially important inareas such as X-ray and far infrared imaging in which lenses are veryexpensive.

SUMMARY OF THE EMBODIMENTS

Embodiments of the present invention improve upon prior art methods byusing a minimal number of masks specially selected for excellentspectral phase and thereby complete information recovery. Consequently,the speed of computation is also increased. According to one method forrecovering phase information from an array of points (for example,pixels) each having an amplitude, at least one transformation unithaving an input and a spectral output is provided. The array of pointsmay arrive optically in a coherent wave or electronically as data. Thearray may be one dimensional or higher dimensional, with two dimensionalapplications being more common. Amplitude information is recorded at thespectral points. The transformation unit may be a lens system with oneor more lenses, or free space wave propagation, or a digital processingunit.

Acting upon the input to the transformation unit are at least twospecially selected masks. There are two masking versions. In the firstversion, one of the masks is a unity mask (also referred to as atransparent mask with all its elements equal to 1). In the secondversion, there is at least one pair of complementary unipolar masks withtheir elements equal to 0 or 1 in amplitude. The input is appliedseparately to each of the at least two masks to generate a modifiedinput from each of the masks. In accordance with an optical embodimentor the like, the masks are physical spatial masks. In such anembodiment, the input is a wave. The mask operating on the wave can beswitched from one of the masks to another such that the input isindividually received separately in sequence by each of the at least twophysical spatial masks. Such switching can be accomplished in real timeby optical devices such as a spatial light modulator or micromirrorarray, for example. Alternatively, the input wave could be split so thatit is individually received in parallel by each of the physical spatialmasks.

In any of the embodiments, it can be advantageous to include an outerborder surrounding each mask that sets amplitudes of any points thatcoincide with the border to zero.

In accordance with embodiments of the invention, the number of masksrequired can be reduced to two or three. In one embodiment, the masksconsist of the unity mask and a phase mask (FIG. 1C-I). In particular,the phase may involve quantized phase values. Thus, in a particularembodiment the phase mask is a bipolar (meaning 1 and −1) binary maskcorresponding to phase values equal to 0 or π (FIG. 1C-II). In a furtherembodiment, the masks consist of the unity mask and at least one pair ofcomplementary unipolar binary masks (with one mask having elements 1 and0, and the other one having 1's and 0's switched) (FIG. 1C-III)Moreover, the masks may consist of the unity mask and a pair of masksthat are complementary with respect to amplitude equal to 1.

Efficient selection of masks can also be achieved in embodiments that donot include use of a unity mask. In accordance with this still furtherembodiment, there are four masks including two pairs of masks (FIG. 2).In each pair, the masks are complementary with each other with respectto amplitude. Thus, unity elements on such a mask may further include aphase factor in which the phase may involve continuous values between 0and 2π, or quantized phase values. In a more particular embodiment, themasks consist of two pairs of complementary unipolar binary masks.

A generalized Fourier transform (FT) as that term is used hereinencompasses a transform performed physically or digitally. A generalizedFT is performed by the transformation unit on the modified inputsreceived from each mask to produce transformed modified inputs. Thespectral plane (output) is defined as the output (plane) of thegeneralized FT. The generalized FT naturally occurs due to coherent wavepropagation and/or when the modified inputs pass through a lens system.It involves additional phase factors. A prominent example is Fresneldiffraction in coherent optics.

For a transformation unit that is a digital processing unit, thegeneralized FT may be a generalized fast Fourier transform (FFT). At thespectral output of the transformation unit, amplitude values arerecorded at an array of points to produce a phasorgram from each of thetransformed modified inputs. In optical embodiments, the recording canbe done by an intensity sensor, such as a camera at the spectral plane(output) of the lens system. The resulting amplitude information on thespectral output is called a phasorgram.

The method further includes associating a phase value with each point oneach phasorgram to form a plurality of complex phasorgrams. The phasevalue may initially be a random phase value in any of the embodiments.The complex phasorgrams are fed into an iterative process that runsuntil convergence is achieved to produce a totagram constituting areconstructed input with amplitude and phase information. The totagramincludes complete and valuable information that can be used in anynumber of ways. For example in any of the embodiments, the totagram canbe used to display a representation of the reconstructed input withamplitude and phase.

In accordance with an embodiment for performing the iterativeprocessing, the plurality of complex phasorgrams are processed by aninverse generalized Fourier transform and possibly other optimizationsteps. A single estimate of the input is obtained by averaging thecomplex information at each input point. The single estimate of theinput is passed through a process replicating each of the masks toobtain a plurality of intermediate arrays of points. A generalized fastFourier transform is performed on each of the intermediate arrays, andthen the amplitude values at each point in the transformed intermediatearrays are replaced with the corresponding initially recorded amplitudevalues to generate another plurality of complex phasorgrams. There maybe additional optimization steps here. The iterative process is repeatedwith the generated complex phasorgrams until convergence is achieved,wherein upon completion the single estimate of the input is thetotagram.

In any of the embodiments, any number of methods can be used todetermine convergence. A simple method is to count up to a given numberof iterations. Alternatively, convergence is achieved when an absolutedifference between successive single estimates reaches a predeterminedthreshold.

In any of the optical embodiments, the at least one transformation unitincludes a lowpass filter that has a numerical aperture (NA) that isequal to or greater than 0.7.

Any embodiment may generate superresolved amplitude and phaseinformation of the input wavefront by either applying linear phasemodulation to the input wave prior to passing the input wave througheach of the at least two physical spatial masks or by moving theintensity sensor spatially.

Any embodiment may include performing a preceding generalized Fouriertransform (FT) on the input prior to separately applying the input toeach of the at least two masks, for example, for lensless imaging ofdistant objects.

In any embodiment, the at least two physical spatial masks may haveelements, each with an aperture size of one of (i) 8×8 pixels or less,or (ii) 16×16 pixels or less for easier implementation. Each element ofa mask has an associated constant amplitude and/or phase that is appliedto each of the pixels or points passed through that element of the mask.

A system embodiment of the invention operates in accordance with one ormore of the method embodiments. The system includes a transformationunit, which may be a lens system with one or more lenses or may be adigital processing unit. The system further includes at least two masks.In accordance with some embodiments, the at least two masks include aunity mask. In one embodiment, the masks consist of the unity mask and aphase mask (FIG. 1C-I). The phase mask may have quantized phase values.In particular, the masks may consist of the unity mask and a bipolarbinary mask (FIG. 1C-II). In a further embodiment, the masks consist ofthe unity mask and a pair of complementary unipolar binary masks (FIG.1C-III). More generally speaking, the masks may consist of the unitymask and one pair of masks or more than one pair of masks that arecomplementary with respect to binary amplitude.

In accordance with still further embodiments, the masks may consist ofone pair or more than one pair of masks that are complementary withrespect to amplitude without a unity mask. In particular, there may befour masks including two pairs of masks, wherein the masks in each pairare complementary with each other with respect to amplitude. This meansthe values 1 and 0 in one mask become 0 and 1, respectively, in thesecond mask (FIG. 2). Unity points on such a mask may further include aphase factor. In a more particular embodiment, the masks consist of twopairs of complementary unipolar binary masks.

In any of the embodiments, it can be advantageous to include an outerborder surrounding each mask that sets amplitudes of any points thatcoincide with the border to zero. Indeed, according to a still furtherembodiment, the masks consist of one pair of complementary unipolarbinary masks each with an outer border that sets amplitudes of anypoints that coincide with the border to zero.

In optical embodiments, the masks are physical spatial masks disposed atthe input plane of the optical lens system. The masks operating on thewave can be switched from one of the masks to another such that theinput is individually received separately in sequence by each of the atleast two physical spatial masks. Such switching can be accomplished inreal time by optical devices such as a spatial light modulator ormicromirror array, for example. Alternatively, the input wave can besplit by a beam splitter so that it is individually received in parallelby each of the physical spatial masks.

An input separately modified by each of the masks is passed through thetransformation unit. The amplitude values at an array of points of thetransformed modified inputs are recorded to produce phasorgrams. In theoptical embodiments, recording of amplitude values is performed by atleast one sensor system. The sensor system may be an intensity sensor,such as a camera.

The system further includes a processor configured to (1) associate aninitial phase value with each point on each phasorgram to form aplurality of complex phasorgrams; and (2) iteratively process theplurality of complex phasorgrams until convergence is achieved toproduce a totagram constituting a reconstructed input with amplitude andphase information.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing features of embodiments will be more readily understood byreference to the following detailed description, taken with reference tothe accompanying drawings, in which:

FIG. 1A is a schematic view of an embodiment of a system, in accordancewith the present invention.

FIG. 1B-I is a schematic view of an embodiment of another system, inaccordance with the present invention.

FIG. 1B-II is a schematic view of an embodiment of another system, inaccordance with the present invention.

FIG. 1C-I is a schematic view of an embodiment of a system having aunity mask and a phase mask, in accordance with the present invention.

FIGS. 1C-II is a schematic view of an embodiment of a system having aunity mask and a bipolar binary mask, in accordance with the presentinvention.

FIGS. 1C-III is a schematic view of an embodiment of a system having aunity mask and a pair of unipolar masks, in accordance with the presentinvention.

FIG. 2 is a schematic view of an embodiment of a system having two pairsof unipolar masks, in accordance with the present invention.

FIGS. 3A and 3B is a flow diagram of a method of phase recovery, inaccordance with embodiments of the present invention.

FIG. 4 is a binary spatial mask when the aperture size is 16×16 pixels.

FIG. 5 is a binary spatial mask when the aperture size is 8×8 pixels.

FIG. 6 shows the reconstruction results using G2 with one unity mask andone bipolar binary mask.

FIG. 7 shows the error reduction curve with one unity mask and onebipolar binary mask when the aperture size is 16×16 pixels.

FIG. 8 shows reconstruction results using G2 with one pair ofcomplementary unipolar binary masks.

FIG. 9A shows the reconstruction results using 2 bipolar binary maskswith the Fienup iterative phase recovery method.

FIG. 9B shows the reconstruction results using 3 bipolar binary maskswith the Fienup iterative phase recovery method.

FIG. 10 shows the reconstruction results of an embodiment of theinvention using one unity mask and one bipolar binary mask with theFienup iterative phase recovery method.

FIG. 11 is the reconstruction results of an embodiment of the inventionusing two pairs of complementary unipolar binary masks with the Fienupiterative phase recovery method.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

Definitions. As used in this description and the accompanying claims,the following terms shall have the meanings indicated, unless thecontext otherwise requires:

The term “totagram” is defined herein as the resulting input phase andamplitude information from the iterative spectral phase recovery processusing masks. The information can be one dimensional ormulti-dimensional. In particular embodiments, the totagram is thereconstructed amplitude and phase of an input coherent wave at aparticular wavelength.

The term “totagraphy” or the “totagraphic method” herein is definedherein as the process of obtaining totagrams.

“Totagraphic imaging” involves recording of spectral amplitude by asensor/camera on the spectral plane in contrast to other imaging systemswhere recording of image information is done by a camera on the imageplane.

“Holography” involves a physical recording of an interference patterndue to mixing of an object wave and a reference wave creating ahologram. On the other hand, totagraphy replaces the recording of aninterference pattern between an object wave and a reference wave, as inholography, but instead performs several measurements using specialmasks which are iteratively processed to create a totagram, using themethods and systems defined herein.

A “phasorgram” is defined herein as information that includes themeasured or recorded spectral amplitude information after processing aninput wave by the transformation unit (e.g., a generalized Fouriertransform) with respect to a particular input mask. Phasorgrams havelittle or no resemblance to the input wave because the spectral phaseinformation is discarded and spectral amplitude is recorded.

INTRODUCTION

FIG. 1A is a schematic view of an embodiment of a system 100 for usewith the present invention. The system 100 recovers phase and amplitudeinformation from a coherent input wave 102. The input wave 102 may begenerated from an object 120. The object 120 may be an illuminatedobject. The system 100 includes at least one transformation unit 110having a spectral plane (SP) and input plane (IP) and at least two masks108. The system is configured so that the input wave 102 is separatelyapplied in series to each of the masks. In optical embodiments, themasks are physical spatial masks. Such physical masks may be implementedto change in real time from one mask to another by optical devices suchas a spatial light modulator or a micromirror array. The at least twophysical spatial masks may be located at the IP of the at least onetransformation unit 110. In some embodiments, the transformation unitmay be a lens system having one or more lenses. In any of the opticalembodiments, the at least one transformation unit may also function as alowpass filter that has a numerical aperture (NA) that is equal to orgreater than 0.7. In other embodiments, the transformation unit may beimplemented in a digital processor.

Each of the at least two masks may include an input window 106 formed ofa respective opaque border surrounding the mask. Each opaque border isconfigured to block pixels in the input wave coinciding with the borderthereby setting amplitudes of those pixels to zero. The at least twomasks 108 are configured to modify phase or amplitude of its separatelyreceived input wave. The at least one transformation unit is configuredto perform a generalized Fourier transform (FT) on the modifiedseparately received input wave.

The system 100 further includes at least one sensor 112 configured torecord amplitude values at an array of points of each transformedmodified input at the SP. The at least one sensor generates a phasorgram152 that includes the measured or recorded spectral amplitudeinformation. Phasorgrams 152 may have little or no resemblance to theinput wave 102 because the phase information is discarded. The sensor112 may be a camera, which is an intensity sensor. The amplitude valuesare directly derived from intensity. Intensity is understood to belinearly proportional to the square of amplitude.

The system 100 further includes a digital processor 128. Phasorgrams 152are iteratively processed by the processor 128 to generate a totagram158. The processor 128 is configured to: associate a phase value witheach point on each phasorgram to form a plurality of complexphasorgrams; and iteratively process the plurality of complexphasorgrams until convergence is achieved to produce a totagram 158constituting a reconstructed input wave with amplitude and phaseinformation. The spectral phase is recovered to go along with therecorded amplitude values. The input amplitude and phase can be obtainedfrom the spectral phase and amplitude through using a generalized IFFT,if desired. The processor 128 may provide the totagram 158 for furtherprocessing 162. The computer processing 162 may include imageprocessing, machine learning and/or deep learning. The processed result178 may form an image 172 in a display 170 that is accessible by a userinterface 118.

FIG. 1B-I and FIG. 1B-II are schematic views of systems 150A and 150B,respectively, in accordance with embodiments for separately applying theinput in parallel to a plurality of physical spatial masks. The systems150A and 150B each recover phase and amplitude information from an inputwave 102. The input wave 102 may be generated from an object 120. Theobject 120 may be an illuminated object. The systems 150A and 150B eachinclude at least two physical spatial masks 108 each disposed at theinput plane of the corresponding transformation unit 110.

The systems 150A and 150B each further include a splitter 130A (alsoknown as “beamsplitter” herein) configured to split the input wave 102into two or more separate waves. Each of the separate waves from thesplitter passes through a corresponding one of the at least two physicalspatial masks 108 to produce a modified wave. The at least onetransformation unit 110 is configured to perform a generalized Fouriertransform (FT) on the modified input wave 102. The systems 150A and 150Binclude a sensor 112 configured to record spectral amplitude images ofthe transformed separate waves at the spectral plane for eachtransformation unit. The systems 150A and 150B include a processor 128which operates as previously described with respect to FIG. 1A.

In addition, system 150B, as shown in FIG. 1B-II, may perform, via atransformation unit 107, a preceding generalized Fourier transform (FT)on the input wave 102, prior to passing the input wave 102 individuallythrough the splitter 130A. In particular embodiments, the transformationunit 107 is a lens that receives the input wave enroute to the inputmasks 108. The preceding generalized Fourier transform of transformationunit 107 converts the initial input plane image (wave) to a second image(wave). On this plane, the input masks 108 are used with the secondimage with a generalized Fourier transform through transformation unit110 as before for illuminating the intensity sensor 112.

The iterative phase recovery process includes only the second image(wave). Once the phase recovery is completed, the initial input planeimage (wave) is recovered by a final inverse generalized Fouriertransform.

In accordance with embodiments of the present invention, each of FIGS.1C-I, 1C-II, 1C-III, and 2 illustrate minimal sets of masks that achievecomplete phase recovery more quickly and efficiently than systems withmany more masks. FIGS. 1C-I, 1C-I and 1C-III make advantageous use of aunity mask 108 t, which can also be referred to as a transparent mask.The input wave passes through the unity mask undisturbed, thusphysically the unity mask may be achieved by any unobstructed lightpath. Optionally, to further improve on the efficiency of thecomputational process an opaque outer border may surround the unity maskas shown in the figures. The border sets amplitude of points on the wavecoinciding with the outer border to zero. While multiple masks can beadded to a system, according to these embodiments of the invention,inclusion of a unity mask can achieve the desired totagram with as fewas one or two additional masks. If more masks are used, the finalinformation recovery is of higher quality when one of the masks is aunity mask.

FIG. 1C-I is a schematic view of an embodiment of a system having aunity mask 108 t and a phase mask 108 c. A phase mask 108 c imparts aphase shift on points passing through the mask. Each pixel or element(group of pixels) of the mask may impart its own designated phase shiftwhich may vary from element to element. In a two-dimensional mask, anelement is typically a square of pixels having an aperture size measuredby the number of pixels on a side. A phase mask may be advantageouslysimplified by involving quantized phase values. For example, with2-level quantization, the quantized phase values are 0 and π, resultingin phase factors equal to 1 or −1. Such a phase mask is called a bipolarbinary mask.

As shown in FIG. 1C-II, a unity mask 108 t and a bipolar binary mask 108b can be the only two masks, in accordance with an embodiment of thepresent invention. Even these two simple masks efficiently produce atotagram. The unity mask is essentially all 1's and the bipolar binarymask is a “checkerboard” of randomly distributed 1's and −1's. Opaqueouter borders can surround the masks for further improving efficiency.

FIG. 1C-III is an embodiment of the present invention that takesadvantage of unipolar binary masks 108 u. The pixels or elements of aunipolar binary mask are either open (an amplitude of 1 meaning pass) orclosed (an amplitude of 0 meaning no pass). The pixels or elements arearranged in a random pattern. The unipolar binary masks are used in apair in which the masks are complementary with each other with respectto amplitude. This means for an element location in one mask having anamplitude of 1, the corresponding element in the other mask has anamplitude of 0. A minimal mask configuration for efficiently generatingtotagrams includes a unity mask 108 t and a complementary pair ofunipolar binary masks 108 u.

FIG. 2 is a schematic view of an alternative embodiment of the presentinvention that uses two pairs of complementary binary masks 108 v,wherein a unity mask is not necessary. In accordance with thisembodiment a totagram can be determined using at least four masks. Thefour masks include two pairs of masks, wherein the masks in a pair arecomplementary with each other with respect to amplitude. In particular,the four masks may all be unipolar binary masks. While the amplitudes ofthe mask elements are 1's or 0's, it is also possible to include randomphase factors in the 1 elements making the masks complex.

In a still further embodiment of the present invention, the masks 108 vcan be reduced to only one pair of complementary binary masks. Such aconfiguration of masks may have difficulty producing a totagram when theinput has a full range of phase variation from 0 to 2π. But for inputslimited in phase in a narrower range such as between 0 and π, one pairof complementary masks can be sufficient. Again, the masks in a pair arecomplementary with each other with respect to amplitude. In particular,both masks may be unipolar binary masks. This case would be furtherimproved if opaque borders surround the masks. Thus, for certainapplications, this single pair of complementary binary masks may be usedinstead of two pairs.

The phase recovery system for use with the masks of embodiments of thepresent invention shall now be described in greater detail with respectto FIGS. 3A and 3B. The input 102 to the system is an array of points,each point having an amplitude. The array may be one dimensional orhigher. In an optical environment, the input is a coherent wave capturedas a two-dimensional array of pixels (points). The system may receiveits input via an input image (wave) from an object 120. The object 120may be an illuminated object. To recover full color wave information,several phase recovery systems can be configured to run in parallel. Forexample, each system can operate on its own coherent wave for one of thethree primary colors (wavelengths). This can be generalized tomultispectral and hyperspectral images (waves) with more than 3wavelengths.

The input needs to be separately presented to each of the masks. This iseasily performed in a digital embodiment processing the input arrayseparately through each of a plurality of masks. In an opticalembodiment, a splitter 130B can be used to replicate the input pointarray for each of the masks. Alternatively, the input masks can beswitched out in series by a spatial light modulator or micromirror arrayas described with respect to FIG. 1A.

The system is configured with a plurality of masks 188 according to anyof the embodiments described above with respect to FIGS. 1C-I, 1C-II,1C-III and 2. Additional masks can also be used, but it is advantageousto minimize the number of masks and hence the amount of computations andease of implementation. Optionally, outer borders surrounding the maskscan be used to further facilitate efficiency and accuracy of theiterative computational process. The input modified by each mask ispassed through a transformation unit 110 to perform a generalizedFourier transform 182. In optical embodiments, the transformation unitcan be a lens or a system of lenses. In digital embodiments, thegeneralized Fourier transform 182 is computed. The generalized Fouriertransform may be a generalized FFT.

The transformed modified inputs are each fed to a sensor 112 forrecording amplitude values at the spectral array of points of eachtransformed modified input. The array of amplitude values is referred toas a phasorgram. The sensor 112 is insensitive to phase. Thus, any phaseaberrations which can be modeled as phase variations on the spectralplane (output) are removed at the sensor. In optical embodiments, thesensor 112 may be an intensity sensor, such as a camera. Intensity islinearly proportional to the square of amplitude.

The method further includes, in the digital processor 128, associating aphase value 196 with each point on each phasorgram to form a pluralityof complex phasorgrams. In preferred embodiments, a randomly selectedphase value is associated with each point. Inclusion of phase leads to acomplex phasorgram.

The complex phasorgrams enter an iterative process. A number ofapproaches are known in the art. One such process is G2. Otherapproaches are demonstrated in Phasepack, for example. Depending on theprocess being implemented in the system, the complex phasorgrams mayeach optionally go through an optimization process 198 (which can be atthe input and/or the output of the iterative system). Then, each complexphasorgram is processed through an inverse generalized Fourier transform186. For an FFT, the inverse is an IFFT and vice versa.

The outputs of the inverse generalized Fourier transform is optionallyoptimized depending on the process implemented and then the complexinformation at each corresponding point is averaged 178 to produce asingle estimate of the input. Another optimization process may beoptionally included at the output side of the iterative process. Eachtime a single estimate is obtained, the process determines whetherconvergence has occurred 172. According to one convergence test, theprocessing continues until a difference between successive singleestimates reaches a predetermined threshold. According to anotherapproach, convergence is assumed to have been reached after a givennumber of iterations of determining a single estimate have beencompleted. According to some embodiments, the predetermined threshold isreached when the Fractional Error, that is the Sum of the Squared Error(SSE) over all N images output from the inverse generalized Fouriertransforms divided by the amplitudes squared over all N images (thetotal energy) between two successive iterations, is less than a value,such as, but not limited to 0.0001. The SSE represents a differencesquared between the N current waveforms and the last estimate.Alternatively, the SSE can be defined in terms of the current estimateafter averaging and the last estimate. Once convergence has beenachieved, the final estimate of the input amplitude and phaseconstitutes the totagram.

The iterative process is further illustrated in FIG. 3B. The singleestimate is passed through a process 189 replicating each of the inputmasks to obtain a plurality of intermediate arrays, one from each mask.In other words, the phase shift and/or amplitude factor for each elementin the corresponding mask then modifies the input to yield theintermediate arrays. A generalized fast Fourier transform 183 isperformed on each of the intermediate arrays. At each point in thetransformed intermediate array, the amplitude value is replaced 170 bythe corresponding amplitude value initially recorded by the sensors 112,thereby generating another iteration of complex phasorgrams. The complexphasorgrams are optimized, if applicable to the iterative process beingimplemented. Then, as in the first iteration, each complex phasorgram isprocessed through an inverse generalized Fourier transform 186. For anFFT, the inverse is an IFFT and vice versa. The outputs of the inversegeneralized Fourier transform is optionally optimized depending on theprocess implemented and then the complex information at eachcorresponding point is averaged 178 to produce a single estimate of theinput. The process continues iteratively until convergence has occurred172.

Optionally, the phase recovery method may generate superresolvedamplitude and phase information from the input wave by performing linearphase modulation on the input wave a number of times prior to passingthe input wave through each of at least two physical spatial masks ormoving the intensity sensor spatially after passing the input wavethrough each of the at least two physical spatial masks a number oftimes. This can also be achieved by moving the location of the spectraloutput a number of times.

In any embodiment of the at least two masks, each element of a nonunitymask may have an aperture size of 8×8 pixels or less. In any embodimentconsisting of at least two masks, a nonunity mask may have an aperturesize of 16×16 pixels or less. Each element of a nonunity mask has anassociated constant amplitude and/or phase that is applied to each ofthe pixels or points passed through that element of the mask.

Any embodiment may include processing the totagram to provide a solutionto a task. These tasks may include microscopy, encoding, signalprocessing, wavefront sensing, and/or light computing. The informationwithin a totagram can be converted into a hologram by using recoveredamplitude and phase information. The result is known as a digitalhologram or computer-generated hologram. The 3-D information of atotagram can also be visualized in other ways by digital techniques suchas by computer graphics, volumetric displays, virtual reality, augmentedreality, or mixed reality. Any embodiment may include displaying aresult or representation of the solution on a display.

The efficacy of the system and methods of the present invention has beenshown for a wide variety of inputs. If the input has zero phase, thismeans the input has only amplitude variations. This is the simplestcase. The most general case has the input phase varying between 0 and 2πradians.

There are two major categories of suitable mask combinations accordingto the embodiments of the present invention. In the first category, thefirst mask is a unity (clear, transparent, with all elements equal to+1) mask. The second mask can be (1) a phase mask with phase changingbetween 0 and 2π radians, (2) a quantized phase mask with elements equalto quantized phase values, (3) a bipolar binary mask with elements equalto +1 and −1, corresponding to quantized phases chosen as 0 and piradians, (4) a pair of complementary masks, meaning one mask haselements 0 and exp(jθ₁), θ₁ being a quantized or continuous phase, andthe second mask having corresponding elements equal to exp(jθ₂), θ₂being a quantized or continuous phase, and 0, respectively. In otherwords, the masks are complementary with respect to amplitude. If anelement of one mask has the value 0, the corresponding element in theother mask of the pair has amplitude of 1 and the associated phasefactor. In a specific case, when θ₁ and θ₂ are chosen equal to 0, themasks become a complementary pair of unipolar binary masks with elementsequal to 0 and 1. In another specific case, when θ₁ and θ₂ are limitedto either 0 or π, the masks become a complementary pair of binary maskswith elements equal to 0 and ±1. Binary refers to the two amplitudevalues, either 0 or 1.

In the second category, the transparent mask is not required, rather,there are pairs of complementary masks, preferably two or more pairs. Inparticular, two pairs of complementary unipolar (+1 and 0) binary maskscan be effectively used. If more masks are used, the number of phaserecovery iterations are usually reduced.

In all cases discussed in categories 1 and 2, it is possible to useouter borders filled with zeros. Use of borders, for example, bydoubling the mask size and filling the outer border of the mask withzeros usually gives more accurate reconstruction results or reducednumber of phase recovery iterations.

COHERENT PHASE/AMPLITUDE RECOVERY WITH G2

A major application of coherent phase/amplitude recovery is imagingwhich can be 2-D, 3-D or higher dimensional. In order to achievemultidimensional imaging, it is necessary to have complete waveinformation consisting of amplitude and phase. Below G2 is discussed asan example of a number of candidate methods for coherent phase/amplituderecovery.

Assuming a constant z (the longitudinal direction), the coherent spatialwave can be written as

u(x,y)=A(x,y)e ^(jα(x,y))  (1)

where A(x,y) is the input spatial amplitude and α(x,y) is the inputspatial phase at (x, y, z).

At this point, we will assume that the wave is generalized Fouriertransformed. In a digital implementation, this means the wave isprocessed by generalized FFT. In an optical implementation, the wavegoes through a lens system with focal length F. Then, the initial waveis assumed to be at z=−F. The spectral plane is at z=F. It is known thaton the spectral plane, the wave is proportional to the Fourier transformof the input wave [O. K. Ersoy, Diffraction, Fourier Optics and Imaging,J. Wiley, November 2006, incorporated by reference in its entiretyherein]. This is the case discussed below.

On the spectral plane, the corresponding wave can be written as

U(f _(x) ,f _(y))=B(f _(x) ,f _(y))e ^(jθ(f) ^(x) ^(,f) ^(y) ⁾  (2)

where B(f_(x),f_(y)) is the spectral amplitude, and θ(f_(x),f_(y)) isthe spectral phase. (f_(x),f_(y)) corresponds to the spatialfrequencies. With the lens system, they are given by

f _(x) =x _(f) /λF  (3)

f _(y) =y _(f) /λF  (4)

where λ is the wavelength, and (x_(f),y_(f)) are the spatial coordinateson the spectral plane.

Assuming the sensor is located on the spectral plane, or on purpose, thespectral phase is lost, and the spectral amplitude is obtained viaspectral intensity I(f_(x),f_(y)) as

I(f _(x) ,f _(y))=|B(f _(x) ,f _(y))|²  (5)

In subsequent iterations with a computer, I(f_(x),f_(y)) is processedfurther by fast Fourier transform (FFT) techniques.

Below the details of digital processing with the discrete Fouriertransform (DFT) and its inverse (IDFT), their fast algorithms fastFourier transform (FFT) and inverse fast Fourier transform (IFFT) arefurther described. The following will be defined:

S: input signal

P_(i): input mask, i=1, 2 . . . , M

M: number of masks

FT: Fourier transform (DFT in numerical work)

IFT: Inverse Fourier transform (IDFT in numerical work)

θ_(i)=output phase, i=1, 2 . . . , M

θ_(i) is chosen randomly in the range [0, 2π] in the first iterationduring phase recovery.

The initial transformations in the first iteration between the inputspace and the output space are as follows:

S _(i) =P _(i) ·S, i=1, 2 . . . , M  (6)

A _(i)=|FT(S _(i))|, i=1, 2 . . . , M  (7)

U _(i)=IFT(A _(i) ·e ^(jθ) ^(i) ), i=1, 2 . . . , M  (8)

V _(i) =U _(i) /P _(i) , i=1, 2 . . . , M  (9)

where the operations · and / denote pointwise multiplications anddivisions, respectively: The next iteration is started after averagingV_(i)'s as follows:

$\begin{matrix}{S = {\frac{1}{M}{\sum\limits_{i = 1}^{M}V_{i}}}} & (10)\end{matrix}$

Then, equations (6-10) during the current iteration are repeated. Theiterations are stopped either by checking whether A_(i)·e^(jθ) ^(i) ischanging negligibly or if a specified maximum number of iterations arecompleted.

The DFT and inverse DFT in the 1-D case are given by

$\begin{matrix}{{{S_{2}^{i}(k)} = {\sum\limits_{n = 0}^{N - 1}{{S_{1}^{i}(n)}e^{{- j}\; 2\pi\;{nk}\text{/}N}\mspace{14mu} n}}},{k = 0},1,2,\ldots\mspace{14mu},( {N - 1} )} & (11) \\{{{S_{i}(k)} = {\frac{1}{N}{\sum\limits_{n = 0}^{N - 1}{{S_{3}^{i}(n)}e^{j\; 2\pi\;{nk}\text{/}N}\mspace{14mu} n}}}},{k = 0},1,2,\ldots\mspace{14mu},( {N - 1} )} & (12)\end{matrix}$

Equations (11) and (12) can be easily extended to the 2-D case.

DESIGN FOR DIGITAL/OPTICAL IMPLEMENTATION

Digital implementation of iterative phase recovery methods can be donein a computer system.

Digital/optical implementation of the iterative phase recovery methodscan also be done by fabrication of an optical system to be coupled witha digital system fed by the output of the digital sensor/camera forsubsequent iterative processing.

For digital/optical implementation, spectral imaging with a highresolution camera and real time electronic phase/amplitude masks such asspatial light modulators are used. Subsequent digital processing is doneby a computer system with high precision. FFT techniques require theirown sampling intervals. These should be matched to the pixel intervalswith the camera.

Once amplitude and phase recovery is completed in the optical/digitalsystem, the information is called a totagram.

Any embodiment may include processing the totagram to provide a solutionto a task. These tasks may include microscopy, encoding, signalprocessing, wavefront sensing, and/or light computing. The informationwithin a totagram can be converted into a hologram by using recoveredamplitude and phase information. The result is known as a digitalhologram or computer-generated hologram. The 3-D information of atotagram can be visualized in other ways by digital techniques such asby computer graphics, volumetric displays, virtual reality, augmentedreality, or mixed reality.

The experimental results in an optical/digital system may not be asperfect as the purely digital implementation results. In order tocompensate for the differences, machine learning (ML) and deep learning(DL) techniques can be used to improve the results. Such techniques haverecently been reported for aiding phase recovery and diffractive imaging[Y. Rivenson, Y. Zhang, H. Günaydin, Da Teng and A. Ozcan, ‘PhaseRecovery and Holographic Image Reconstruction Using Deep Learning inNeural Networks,’ Light: Science & Applications, Vol. 7, 17141, 2018,incorporated by reference in its entirety herein]. G. Barbastatis, A.Ozcan, G. Situ, ‘On the Use of Deep Learning for Computational Imaging,’Optica, Vol. 6, No. 8, pp. 921-943, August 2019, incorporated byreference in its entirety herein]. ML and DL utilize very large databases of images. For example, the input image to the system can be whatis achieved experimentally, and the output desired image is what itshould ideally be. By training with a very large database of suchimages, ML and DL methods have been reported to achieve good results.

ITERATIVE PHASE RECOVERY METHODS WITH DIFFRACTION LIMITED OPTICALCOMPONENTS

The transformation unit 110 may be a coherent optical system that is atleast diffraction-limited, and is governed by a point-spread functionand its Fourier transform, the coherent transfer function (CTF). Thesystem acts as an ideal lowpass filter with a cutoff frequency governedby the lens system numerical aperture NA. In this section, we show andclaim that with sufficiently large NA (˜0.7), iterative phase recoveryis unhindered by diffraction.

A diffraction limited lens system acts as a linear system with a pointspread function h(x,y) and a coherent transfer function H(f_(x),f_(y))which is the Fourier transform of h(x,y). The linear system equation inthe space domain is given by

u _(output)(x,y)=h(x,y)*u _(input)(x,y)  (13)

where * denotes linear 2-D convolution, and u_(output)(x,y) is theoutput spatial wave. The corresponding spectral equation by convolutiontheorem is given by

U(f _(x) ,f _(y))=H(f _(x) ,f _(y))U(f _(x) ,f _(y))  (14)

where U(f_(x),f_(y)) is the Fourier transform of the output spatialwave.

A coherent wave illumination on a 3-D object will be assumed. This canbe achieved with a laser or high quality light emitting diode (LED). Forexample, a He-Ne laser operates at wavelength λ equal to 0.6386 micron(μ=10⁻⁶ m), and a LED operates around λ=0.5μ.

Some quantities of interest are the following:

-   -   k₀=2π/λ, the wave number    -   NA=numerical aperture    -   sp=sampling pixel size of camera    -   fp=final pixel size of reconstruction

Due to diffraction, the optical imaging system has a cutoff frequencygiven by

f _(c)=NA·k ₀  (15)

The sampling frequencies on the spectral plane will be written as

kxs=kys=−k _(max) , −k _(max) +Δk, . . . , +k _(max)  (16)

where, for N₁ sampling points along x and y directions, Δk can be chosenas

$\begin{matrix}{{\Delta\; k} = \frac{2k_{\max}}{N_{1} - 1}} & (17)\end{matrix}$

Then, the coherent transfer function is given by

$\begin{matrix}{{H( {f_{x},f_{y}} )} = {{{CTF}( {f_{x},f_{y}} )} = \{ {\begin{matrix}{{1\mspace{11mu} k_{s}} < f_{c}} \\{0\mspace{14mu}{otherwise}}\end{matrix}{where}} }} & (18) \\{k_{S} = \sqrt{{kxs}^{2} + {kys}^{2}}} & (19)\end{matrix}$

for each component of kxs and kys.

The inventor has made experiments to discover which values of NA allowsfor a perfect reconstruction. It has been determined that with NA=0.7 orhigher the reconstructed images are visually as good as the originals.

ABERRATIONS

Aberrations are departures of the ideal wave within the exit pupil of alens system from its ideal form. In a coherent imaging system, this canbe modeled as multiplying the optical transfer function by a phasefactor. In this section, it is shown and claimed that phase aberrationshave no detrimental effect on the performance of iterative phaserecovery methods.

A diffraction-limited system means the wave of interest is perfect atthe exit pupil, and the only imperfection is the finite aperture size.Aberrations are departures of the ideal wave within the exit pupil fromits ideal form. In order to include phase aberrations, the exit pupilfunction can be modified as

P _(A)(x,y)=P(x,y)e ^(jkϕ) ^(A) ^((x,y))  (20)

where P(x,y) is the exit pupil function without aberrations, andϕ_(A)(x,y) is the phase error due to aberrations.

The phase function ϕ_(A)(x,y) is often written in terms of the polarcoordinates as ϕ_(A)(r,θ). What is referred to as Seidel abberations isthe representation of ϕ_(A)(r,θ) as a polynomial in r, for example,

ϕ_(A)(r,θ)=a ₄₀ r ⁴ +a ₃₁ r ³ cos θ+a ₂₀ r ² +a ₂₂ r ² cos² θ+a ₁₁ r cosθ  (21)

Higher order terms can be added to this function. The terms on theright-hand side of Eq. (19) represent the following:

-   -   α₄₀r⁴: spherical aberration    -   a₃₁r³ cos θ: coma    -   a₂₀r²: astigmatism    -   a₂₂r² cos² θ: field curvature    -   a₁₁r cos θ: distortion

Zernicke Polynomials

The phase aberrations present in an optical system can also berepresented in terms of Zernicke polynomials, which are orthogonal andnormalized within a circle of unit radius [V. N. Mahajan, “Zernikecircle polynomials and optical aberrations of systems with circularpupils,” Engineering and Laboratory Notes, R. R. Shannon, editor,supplement to Applied Optics, pp. 8121-8124, December 1994]. In thisprocess, the phase function ϕ_(A)(x,y) is represented in terms of anexpansion in Zernike polynomials z_(k)(ρ,θ), where ρ is the radialcoordinate within the unit circle, and θ is the polar angle.

Each Zernike polynomial is usually expressed in the form

z _(k)(ρ,θ)=R _(n) ^(m)(ρ)cos mθ  (22)

where n,m are nonnegative integers. R_(n) ^(m)(ρ) is a polynomial ofdegree n, and contains no power of n less than m. In addition, R_(n)^(m)(ρ) is even (odd) when m is even (odd), respectively. Therepresentation of ϕ_(A)(x,y)=ϕ_(A)(ρ,θ) can be written as

$\begin{matrix}{{\phi_{A}( {\rho,\theta} )} = {A_{\infty} + {\frac{1}{\sqrt{2}}{\sum\limits_{n = 2}^{\infty}{A_{n0}{R_{n}^{0}(\rho)}}}} + {\sum\limits_{n = 1}^{\infty}{\sum\limits_{m = 1}^{\infty}{A_{nm}{R_{n}^{m}(\rho)}{cosm}\;\theta}}}}} & (23)\end{matrix}$

The coefficients A_(nm) are determined for finite values of n and m byleast-squares. In turn, ϕ_(A)(ρ,θ) can also be written as

$\begin{matrix}{{\phi_{A}( {\rho,\theta} )} = {\sum\limits_{k = 1}^{K}{w_{k}{z_{k}( {\rho,\theta} )}}}} & (24)\end{matrix}$

where K is an integer such as 37. The coefficients w_(k) are found byleast-squares. Since each successive Zernike term is orthonormal withrespect to every preceding term, each term contributes independently tothe mean-square aberration. This means the root-mean square error ϕ_(A)due to aberrations can be written as

$\begin{matrix}{\overset{\_}{\phi_{A}} = \lbrack {\sum\limits_{k = 1}^{K}w_{k}^{2}} \rbrack^{\frac{1}{2}}} & (25)\end{matrix}$

Note that the Zernike representation of aberrations is valid when theexit pupil is circular. Otherwise, the Zernike polynomials are notorthogonal.

Coherent optical systems have aberrations. They are usually modeled asphase factors on the spectral plane of the system. For example, suchmodeling can be done in terms of polynomials expressing phase due toaberrations, such as Seidel aberrations and Zernicke polynomials. On thespectral plane, the intensity is measured, and all phase is lost. Thatincludes the phase due to aberrations. The camera eliminates all phase,and consequently phase aberrations which can be represented as phasefactors on the spectral plane have no detrimental effect on theperformance of iterative phase recovery methods according to FIGS. 3Aand 3B.

SUPERRESOLUTION WITH ITERATIVE PHASE RECOVERY METHODS

In the previous section, perfect phase reconstruction was achieved forapplications such as 3-D imaging. This was made possible with a high NAdiffraction-limited lens system, and a high dynamic range, highresolution camera. In this section, a system including linear phasemodulation of the object wave and iterative phase recovery methods isdiscussed to improve a given lens system having low NA, low field ofview and aberrations.

Low NA means filtering out high spatial frequencies on the spectralplane. Low field of view means small area of detection by the camera.Aberrations can be modeled as phase modulation on the spectral plane asdiscussed in the previous section. In order to bypass these problems,and/or to achieve higher resolution than what is possible with the givenlens system and camera, we will consider a method similar to what isused in synthetic aperture microscope [Terry M. Turpin, Leslie H.Gesell, Jeffrey Lapides, Craig H. Price, “Theory of the syntheticaperture microscope,” Proc. SPIE 2566, Advanced Imaging Technologies andCommercial Applications, doi:10.1117/12.217378, 23 Aug. 1995] andFourier ptychographic imaging [G. Zheng, R. Horstmeyer, C. Yang,“Wide-field, high-resolution Fourier ptychographic microscopy,” NaturePhotonics, pp. 739-745, Vol. 7, September 2013]. For this purpose, theinput object wave will be modulated (multiplied) by a number of planewaves given by

u _(input)′(x,y)=u _(input)(x,y)□e ^(j(k) ^(xm) ^(x+k) ^(ym) ^(y)) m=1,2, . . . , M  (26)

This can be achieved in a number of ways. For example, a LED matrixarray can illuminate the 3-D object of interest with angle varied planewaves. Alternatively, a real-time reconfigurable array of diffractiongratings can be generated with spatial light modulators (SLM's) [S.Ahderom, M. Raisi, K. Lo, K. E. Alameh, R. Mavaddah, “Applications ofLiquid crystal light modulators in optical communications,” Proceedingsof 5th IEEE International Conference on High Speed Networks andMultimedia Communications, Jeju Island, Korea, 2002].

For each m, the linear imaging system has an output image given by

u _(output)′(x,y)=h(x,y)*u _(input)′(x,y)  (27)

Assuming a diffraction-limited imaging system, the coherent transferfunction is what governs imaging, and Eq. (24) in the spectral domainbecomes

U′(f _(x) ,f _(y))=H(f _(x) ,f _(y))U(f _(x) −f _(xm) ,f _(y) −f_(ym))  (28)

For example, when the wavelength λ is 0.5 micron, the wave numberbecomes

k ₀=2π/λ=1.23·10⁻⁵ m⁻¹

The cutoff frequency for the CTF is

f _(c)=NA·k ₀  (29)

When NA=0.1, and the DFT (image) size is 256×256, a 32×32 window of DFTspectral points fits right in to the CTF circle with radius f_(c) equalto 1.257·10⁶ m⁻¹. Let K×K be the number of plane waves needed tomodulate the input wave. In this example, we get K=256/32=8.

Similarly, when NA=0.2 and the DFT (image) size is 256×256, a 64×64window of DFT spectral points fits right in to the CTF circle withradius f_(c) equal to 1.257·10⁶ m⁻¹. In this case, for K×K number ofplane waves needed to modulate the input wave, we get K=256/64=4.

Modern high resolution cameras such as 8K cameras support much highernumber of pixels such as 8192×4320 pixels[https://www.usa.canon.com/internet/portal/us/home/products/details/cameras/eos-dslr-and-mirrorless-cameras/dslr/eos-5ds-r].

Since FFT's work best with powers of 2, let us assume a size of4096×4096 pixels. When NA=0.2 and the DFT(image) size is 16384×16384, a4096×4096 window of DFT spectral points fits right in to the CTF circlewith radius f_(c) equal to 1.257·10⁶ m⁻¹. In this case, for K×K numberof plane waves needed to modulate the input wave, we get K=16384/4096=4.In other words, this system would achieve superresolution with16384×16384 pixels. By using all K×K plane waves, the amplitudes at16384×16384 pixels are obtained with each input mask. The rest isprocessing with iterative phase recovery method iterations.

MASKS

A major consideration is how many masks are needed for an iterativephase recovery method to result in acceptable performance. Since eachmask means another set of measurements, the fewer masks the better. Inaddition, the masks used seriously affect the quality of informationreconstruction. Recovery of information can be considered in terms ofinput image amplitude recovery, phase recovery, or preferably both. Thisis different from phase recovery in the spectral domain. In other words,recovered phase in the spectral domain may give correct input imageamplitude recovery, but not necessarily correct input image phaserecovery or incomplete input image phase recovery. In the literature,what is usually reported is the input (wave) amplitude recovery. It ishighly probable that the recovered input (wave) phase is notsufficiently correct. In accordance with embodiments of the presentinvention, complete input image amplitude recovery as well as inputphase recovery is sought.

Another consideration is type of masks to be used. Reducing the numberof masks is achievable in accordance with embodiments of the presentinvention. As explained above, in the first category, the first mask isa unity (clear, transparent, with all elements equal to +1) mask. Thesecond mask can be (1) a phase mask with phase changing between 0 and 2πradians, (2) a quantized phase mask with elements equal to quantizedphase values, (3) a bipolar binary mask with elements equal to +1 and−1, corresponding to quantized phases chosen as 0 and pi radians, (4) apair of complementary masks, wherein corresponding elements of each maskin the pair are complementary with respect to amplitude. In the secondcategory, the transparent mask is not required, rather, there are pairsof complementary binary masks, preferably two or more pairs. Inparticular, two pair of complementary unipolar (+1 and 0) binary maskscan be used. If more number of masks are used, the number of phaserecovery iterations may be reduced.

Embodiments of the present invention implement iterative phase recovery(version 1) with a unity mask in addition to one or more additionalmasks, or (version 2) pairs of unipolar masks complementary with respectto amplitude.

The unipolar binary mask is no longer a phase mask, but a binaryamplitude mask. According to conventional thinking, amplitude masks donot work in general. On the other hand, unipolar binary masks would bedesirable in many applications since they make implementation easier. Inaccordance with embodiments of the invention, unipolar binary masks arecreated in pairs. The second mask is the complement of the first mask.In other words to create the second mask, 0's and 1's are exchanged atevery component of the first mask. This is also the case with pairs ofunipolar binary masks in which 1's are replaced by phase factors whoseamplitudes equal 1.

Each element in the masks has a finite size. So it is important,especially with optical implementations, that finite sized elements donot reduce performance. We claim that the iterative phase recoverymethods function well with finite element sizes as well provided thatthey are sufficiently small. Sufficiently small is 16×16 in the binarybipolar case and 8×8 in the unipolar binary case. A binary mask foraperture size equal to 16×16 pixels is shown in FIG. 4. A binary maskwith an aperture size equal to 8×8 pixels is shown in FIG. 5.

Experimental Results with Complex Waves using the Proposed Masks

Some experimental results are provided with a complex wave havingamplitude (image) and phase (image) in FIGS. 6, 7, and 8. FIG. 6 showsthe reconstruction results with a complex wave using G2 when one unitymask and one bipolar binary mask is used. The original amplitude imageis shown at (a). The reconstructed amplitude image is shown at (b). Theoriginal phase image is shown at (c). The reconstructed phase image isshown at (d). FIG. 7 shows the corresponding error reduction curveduring iterations.

FIG. 8 shows the reconstruction results with the same complex wave usingG2 when one pair of complementary unipolar binary masks is used. Theoriginal amplitude image is shown at (a). The reconstructed amplitudeimage is shown at (b). The original phase image is shown at (c). Thereconstructed phase image is shown at (d).

IMAGING OF DISTANT OBJECTS

In this section, we discuss iterative phase recovery for coherentimaging of objects which are considerably distant from the imaging lenssystem. In such a case, the field at the entrance of the imaging lenssystem is directly related to the Fourier transform of the wave comingfrom a thin object to be imaged. This is especially true in theFraunhofer approximation for distant wave propagation, and can also beextended to not so distant propagation with Fresnel approximation in thefollowing references: A. Eguchi, J. Brewer, T. D. Milster, “Optimizationof random phase diversity for adaptive optics using an LCoS spatiallight modulator,” Optics Letters, Vol. 44, No. 21, 1 Nov. 2019, pp.6834-6840, and A. Eguchi, T. D. Milster, “ Single shot phase retrievalwith complex diversity,” Optics Letters, Vol. 44, No. 21, 1 Nov. 2019,pp. 5108-5111. In previous sections, the input to the lens system wasthe complex image. Now it is essentially a spectral image. We canconsider passing the input wave through the input masks as donepreviously, followed by another generalized Fourier transform, forexample by a lens, which would yield the object image inverted. Then, acamera would record the image. In this geometry, the system is theopposite of the previous systems, meaning the image plane and thespectral plane are exchanged. Unfortunately, iterative phase recoverymay not function well under these conditions. The Fourier transform ofthe object image is usually concentrated at very small frequencies, andthe remainder of the Fourier plane information is noiselike with smallcomponents, making the use of input masks ineffective.

To address these issues, the system shown in FIG. 1B-II can be used forcoherent imaging of thin distant objects. As shown, the lens system maybe designed for providing two Fourier transforms rather than one Fouriertransform. The first Fourier transform 107 converts the input image(wave) to another image (wave). Then, the previous system is used. Theinput is passed through splitter(s) whose outputs are sent to the masksas before. The second Fourier transform 110 regenerates the modifiedspectral information due to masking for illuminating the image sensor112. Thus, the part denoted by the lens system image plane includingmasks and the image sensor plane is the same as the system usedpreviously.

ITERATIVE PHASE RECOVERY METHODS WITH MASKING

A number of iterative phase recovery methods have been used withmasking. According to embodiments of the present invention, using aunity (clear) mask as one of the masks considerably improves theperformance of the iterative phase recovery methods when using bipolarbinary masks or phase masks. Also, the use of pairs of complementarybinary masks in such systems (possibly without unity mask) are highlyeffective.

The effectiveness of an embodiment of the invention was evaluateddigitally. A simple FFT system was used with the digitally implementedmasks without borders. Without losing generality, the coherent input wasan amplitude image only, meaning the input phase is assumed to be zeroat each pixel. FIG. 9A shows image recovery with the Fienup method when2 bipolar binary masks were used. It is observed that the image is notrecovered. The corresponding results with all the other methods were thesame. The results were somewhat improved when using 3 bipolar binarymasks. This is shown in FIG. 9B with the Fienup method. However, theresults are still not satisfactory. Replacing one bipolar binary maskwith a clear mask resulted in drastic improvement as shown in FIG. 10where one clear mask and one bipolar binary mask resulted in imagerecovery.

The results with pairs of complementary unipolar binary masks showedthat they are self-sufficient without a unity mask. FIG. 11 shows imagerecovery results with two pairs of complementary unipolar binary masksusing the Fienup method. It is observed that 2 pairs of complementarymasks produced better results than one pair of complementary masks.Beyond two pairs of masks, the results further improve marginally.

With other iterative phase recovery methods, the performance was verysimilar. Such methods include but are not limited to those listed inTable 1.

TABLE 1 Iterative Phase Recovery Methods. Method Reference WirtFlow(Wirtinger Flow E. J. Candes, Y. Eldar, T. Strohmer, V, Voroninski,“Phase Retrieval Algorithm) via Matrix Completion,” SIAM review, 57(2):225-251, 2015 TWF (Truncated Wirtinger Yuxin Chen and Emmanuel J Candès,“Solving random quadratic Flow Algorithm (with systems of equations isnearly as easy as solving linear systems,” Poisson loss))https://arxiv.org/abs/1505.05114, 2015 RWF (Reweighted Wirtinger ZiyangYuan and Hongxia Wang, “Phase retrieval via reweighted Flow Algorithm)wirtinger flow,” Appl. Opt., 56(9): 2418-2427, March 2017 AmplitudeFlow(Amplitude Gang Wang, Georgios B Giannakis, and Yonina C Eldar, “SolvingFlow Algorithm without systems of random quadratic equations viatruncated amplitude truncation) flow,” arXiv preprint arXiv: 1605.08285,2016 TAF (Truncated Amplitude Gang Wang, Georgios B Giannakis, andYonina C Eldar, “Solving Flow Algorithm) systems of random quadraticequations via truncated amplitude flow,” arXiv preprint arXiv:1605.08285, 2016 RAF (Re-Weighted G. Wang, G. B. Giannakis, Y. Saad, andJ. Chen, “Solving Almost all Amplitude Flow Algorithm) Systems of RandomQuadratic Equations,” ArXiv e-prints, May 2017 GerchbergSaxton(Gerchberg R. W. Gerchberg, W. O. Saxton, “A practical algorithm for theSaxton Algorithm) determination of the phase from image and diffractionplane pictures,” Optik, Vol. 35, pp. 237-246, 1972; R. W. Gerchberg, “ANew Approach to Phase Retrieval of a Wave Front,” Journal of ModernOptics, 49: 7, 1185-1196, 2002 Fienup Algorithm J. R. Fienup, “Phaseretrieval algorithms, a comparison,” Applied Optics, Vol. 21, No. 15,pp. 2758-2769, 1 Aug., 1982 Kaczmarz Algorithm Ke Wei, “Solving systemsof phaseless equations via kaczmarz methods: A proof of concept study,”Inverse Problems, 31(12): 125008, 2015 PhaseMax Algorithm Sohail Bahmaniand Justin Romberg, “Phase retrieval meets statistical learning theory:A flexible convex relaxation,” arXiv preprint arXiv: 1610.04210, 2016;Tom Goldstein and Christoph Studer, “Phasemax: Convex phase retrievalvia basis pursuit,” arXiv preprint arXiv: 1610.07531, 2016

Table 2 shows the mean-square error performance with all the methodswhen using 3 bipolar binary masks versus 1 clear mask and 2 bipolarbinary masks. It is observed that the error performances with large MSEerror (Fienup, G2, TAF, Wirtflow) with no clear mask substantiallyimproved after replacing one bipolar binary mask with a clear mask.

Table 3 shows how the number of iteration and computation time changesas a function of number pairs of complementary unipolar binary maskswith the RAF method, for example. It is observed that the performancegets considerably better in terms of speed of computation as the numberof pairs of masks increases to 3 for that method.

Table 4 shows the optimal number of pairs of complementary unipolarmasks for best visual performance. This number is 2 (mostly) or 3.

TABLE 2 Error performance with 3 masks. MSE Error MSE Error Algorithm(no clear mask) (with clear mask) TWF 7.544e−05  5.36e−06 Fienup 1.341 9.91e−04 G2 1.236  9.95e−04 Amplitudeflow 7.09e−05 7.08e−05 Kaczmarz0.1965 0.134 Phasemax 2.296e+06  3.39e+05 RAF 7.25e−04 2.49e−04 RWF4.84e−04 4.09e−04 TAF 0.0226 5.37e−05 Wirtflow 0.0199  2.1e−04

TABLE 3 Properties as a Function of Pairs of Complementary UnipolarMasks with the RAF method. Number of Mask Pairs 1 2 3 4 Number ofIterations 106 49 22 17 Computation Time (sec) 12.32 8.72 7.25 7.26

TABLE 4 Optimal Number of Pairs of Complementary Unipolar Masks.Algorithm Optimal Number TWF 2 Fienup 2 G2 2 Amplitudeflow 2 Kaczmarz 3Phasemax 2 RAF 3 RWF 1 TAF 3 Wirtflow 1

CONCLUDING REMARKS

Iterative phase recovery methods can be implemented digitally, forexample, within a digital processor, such as a computer. The input mayfor example be a pre-recorded image or other array of points. In thiscase, a generalized FFT and generalized inverse FFT (IFFT) can be used.Using the word ‘optical’ in a general sense to encompass all waves,iterative phase recovery methods can also be implemented by a coherentoptical or by a coherent optical/digital system. In these cases, theinitial Fourier transform operation and amplitude detection is typicallydone by a lens/camera system. In the case of a coherent optical/digitalsystem, wave amplitude information obtained by a lens/camera system isinput to a computer system to carry out the iterations with FFT and IFFTin accordance with the iterative phase recovery method. This can befollowed by possible other operations such as generation of 3-D images.

In a digital implementation, the input masks can be generated within acomputer, possibly together with the complex input information. In acoherent optical or coherent optical/digital implementation, they can beimplemented in real time by optical devices such as spatial lightmodulators and micromirror arrays.

Coherent optical systems are at least diffraction limited. This means alens system acts as a lowpass filter characterized by a numericalaperture NA. Iterative phase recovery functions require that the systemNA is sufficiently large. According to embodiments of the presentinvention, NA≥0.7 was found to be sufficient.

Coherent optical systems have aberrations. They are usually modeled asphase factors on the spectral plane of the system. For example, suchmodeling can be done in terms of polynomials expressing phase due toaberrations, such as Seidel aberrations and Zernicke polynomials. In acoherent system, aberration Phase factors appear as an additional phaseto be added to the input spectral phase on the Fourier plane. The camerais sensitive to amplitude only, eliminating all aberrations which can bemodeled as phase variations on the spectral plane. As such, spectralphase aberrations have no detrimental effect on the performance ofspectral iterative phase recovery methods.

An optical system with limited NA and aberrations can be used to achievesuperresolution by using iterative phase recovery methods and includinglinear phase modulation with the input information a number of times.The linear phase modulation part is like what is done in syntheticaperture microscopy and Fourier ptychographic imaging. Iterative phaserecovery operates with the spectral amplitudes obtained from all thelinearly phase modulated parts of input information with each mask toresult in superresolved amplitude and phase information. Similar resultscan be achieved by moving the intensity sensor spatially instead oflinear phase modulation after passing the input wave through each of theat least two physical spatial masks a number of times.

The input masks can be produced with elements, for example, elementshaving a finite size provided that the sizes are sufficiently small. Inthe case of unipolar binary masks, 8×8 elements or smaller resulted insatisfactory performance in digital experiments. In the case of bipolarbinary masks, 16×16 elements or smaller resulted in satisfactoryperformance in digital experiments. Thus, the bipolar binary masks aremore tolerant than unipolar binary masks. In either case, use of finitesized elements means simpler implementation.

Iterative phase recovery performs well in noise. Images heavilycorrupted by noise can be recovered as they appear in noise. Furtherdenoising can be used to generate clear images.

Coherent distant object imaging can be done with iterative phaserecovery processing. Here the input image (wave) is already Fouriertransformed due to coherent wave propagation and may be compressed.Then, one more Fourier transform generates the decompressed image (wave)information. The rest of the system is the same as what we utilizedpreviously with masks and the iterative process of phase recovery.

When the input is an amplitude image only, phase is zero at each inputpoint. Then excellent results can be achieved without a border regioncomposed of zeros surrounding the input window.

The performance of an iterative phase recovery method is substantiallyincreased by using the claimed methods and systems to reduce thecomputation time, to reduce the number of masks, to reduce the number ofiterations, to increase the quality of reconstruction, and to increasethe ease of implementation by using (1) a unity mask together with oneor more bipolar binary masks with elements equal to 1 and −1, or (2) aunity mask together with one or more phase masks, or (3) a unity masktogether with one pair of masks or more than one pair of masks havingbinary amplitudes of 0's and 1's, in which the masks in the pair arecomplementary to each other with respect to amplitude, or (4) one ormore pairs of complementary masks with binary amplitudes of 0's and 1'swithout needing a unity mask. In all cases, it is possible to use outerborders filled with zeros. Use of borders, for example, by doubling themask size and filling the outer border of the mask with zeros canimprove the result. Using any of these combinations of speciallyselected masks can increase the quality of reconstruction and simplifyimplementation.

POTENTIAL CLAIMS

Various embodiments of the present invention may be characterized by thepotential claims listed in the paragraphs following this paragraph (andbefore the actual claims provided at the end of this application). Thesepotential claims form a part of the written description of thisapplication. Accordingly, subject matter of the following potentialclaims may be presented as actual claims in later proceedings involvingthis application or any application claiming priority based on thisapplication. Inclusion of such potential claims should not be construedto mean that the actual claims do not cover the subject matter of thepotential claims. Thus, a decision to not present these potential claimsin later proceedings should not be construed as a donation of thesubject matter to the public.

Without limitation, potential subject matter that may be claimed(prefaced with the letter “P” so as to avoid confusion with the actualclaims presented below) includes:

P1. A method for recovering phase information from an array of points,each point having an amplitude, the method comprising:

-   -   providing at least one transformation unit having an input and a        spectral output and at least two masks one of which is a unity        mask, each of the at least two masks configured to operate upon        the input of the at least one transformation unit;    -   separately applying the input to each of the at least two masks        to generate a modified input from each of the masks;    -   performing, by the at least one transformation unit, a        generalized Fourier transform on each modified input to produce        transformed modified inputs;    -   recording amplitude values at an array of points of each        transformed modified input to produce phasorgrams;    -   associating a phase value with each point on each phasorgram to        form a plurality of complex phasorgrams; and    -   iteratively processing the plurality of complex phasorgrams        until convergence is achieved to produce a totagram constituting        a reconstructed input with amplitude and phase information.

P2. The method of P1 wherein the array of points is a coherent lightwave.

P3. The method of P1 wherein the array of points is digital data.

P4. The method of P1 or P3 wherein the at least one transformation unitis a generalized Fourier transform process run on a digital processor.

P5. The method of P1, P3 or P4 wherein the at least two masks areimplemented on a digital processor.

P6. The method of P1 or P2, wherein the at least one transformation unitis a lens system.

P7. The method of P1, P2 or P6, wherein the masks are physical spatialmasks and further comprising at least one of:

-   -   switching from one of the at least two physical spatial masks to        another of the at least two physical spatial masks such that the        input is individually received in sequence by each of the at        least two physical spatial masks; and    -   splitting the input wave so that it is individually received in        parallel by each of the at least two physical spatial masks.

P8. The method of P1, P2, P6 or P7, wherein the at least two maskscomprise physical spatial masks implemented in real time by opticaldevices including any of spatial light modulators and micromirrorarrays.

P9. The method of P1, P2, P6, P7 or P8, wherein recording is performedby an intensity sensor.

P10. The method of P1, P2, P6, P7, P8 or P9, further comprising (1)performing linear phase modulation on the input wave prior to passingthe input wave through each of the at least two physical spatial masks anumber of times, or (2) moving the intensity sensor spatially afterpassing the input wave through each of the at least two physical spatialmasks, to generate superresolved amplitude and phase information of theinput wave a number of times.

P11. The method of P1, P2, P6, P7, P8, P9 or P10 further comprisingperforming a generalized Fourier transform on the input through a lensprior to separately applying the input to each of the at least twomasks.

P12. The method of any of the preceding potential claims wherein eachmask includes an outer border that sets amplitude of points coincidingwith the outer border to zero.

P13. The method of any of the preceding potential claims, wherein the atleast two masks consist of the unity mask and a complex phase mask.

P14. The method of P13, wherein the complex phase mask comprises abipolar binary mask.

P15. The method of any of potential claims P1 through P12, wherein theat least two masks consist of the unity mask and a pair of masks,wherein the masks in the pair are complementary with each other withrespect to amplitude equal to 0 or 1.

P16. The method of P15, wherein the pair of masks are complementaryunipolar binary masks.

P17. The method of P15, wherein the pair of masks include unity elementsthat have a phase factor.

P18. The method of any of the preceding claims further comprising, aftercompletion, using the totagram to generate a representation of theinformation embedded in the reconstructed amplitude and phase of theinput.

P19. The method of any of the preceding claims, wherein iterativelyprocessing the plurality of complex phasorgrams comprises

-   -   (a) processing the plurality of complex phasorgrams to obtain a        single estimate of the input by performing on the complex        phasorgrams an inverse generalized Fourier transform and        averaging complex information at each point of the input plane,        and possibly some other additional optimization steps;    -   (b) passing the single estimate of the input through a process        replicating each of the masks to obtain a plurality of        intermediate arrays;    -   (c) performing a generalized fast Fourier transform on each of        the intermediate arrays and replacing amplitude values at each        point in the transformed intermediate arrays with corresponding        recorded amplitude values, and possibly some other additional        optimization steps, to generate another plurality of complex        phasorgrams; and    -   (d) repeating step (a) for the another plurality of complex        phasorgrams followed by steps (b) and (c) until convergence is        achieved, wherein upon completion the single estimate of the        input is the totagram.

P20. The method of P19, wherein convergence is determined by any of (1)when a squared difference between successive single estimates reach apredetermined threshold, and (2) when a given number of iterations ofstep (a) is completed.

P21. A method for recovering phase information from an array of points,each point having an amplitude, the method comprising:

-   -   providing at least one transformation unit having an input and a        spectral output and at least two masks including at least one        pair of masks binary in amplitude, wherein the masks in a pair        are complementary with each other with respect to amplitude,        each of the at least two masks configured to operate upon the        input of the at least one transformation unit;    -   separately applying the input to each of the at least two masks        to generate a modified input from each of the masks;    -   performing, by the at least one transformation unit, a        generalized Fourier transform on each modified input to produce        transformed modified inputs;    -   recording amplitude values at an array of points of each        transformed modified input to produce phasorgrams;    -   associating a phase value with each point on each phasorgram to        form a plurality of complex phasorgrams, and possibly some other        additional optimization steps; and    -   iteratively processing the plurality of complex phasorgrams        until convergence is achieved to produce a totagram constituting        a reconstructed input with amplitude and phase information.

P22. The method of P21 wherein the array of points is a coherent lightwave.

P23. The method of P21 wherein the array of points is digital data.

P24. The method of P21 or P23 wherein the at least one transformationunit is a generalized Fourier transform process run on a digitalprocessor.

P25. The method of P21, P23 or P24 wherein the at least two masks areimplemented on a digital processor.

P26. The method of P21 or P22, wherein the at least one transformationunit is a lens system.

P27. The method of P21, P22 or P26, wherein the masks are physicalspatial masks and further comprising at least one of:

-   -   switching from one of the at least two physical spatial masks to        another of the at least two physical spatial masks such that the        input is individually received in sequence by each of the at        least two physical spatial masks; and    -   splitting the input wave so that it is individually received in        parallel by each of the at least two physical spatial masks.

P28. The method of P21, P22, P26 or P27, wherein the at least two maskscomprise physical spatial masks implemented in real time by opticaldevices including any of spatial light modulators and micromirrorarrays.

P29. The method of P21, P22, P26, P27, P28 or P29, wherein recording isperformed by an intensity sensor.

P30. The method of P21, P22, P26, P27, P28 or P29, further comprising(1) performing linear phase modulation on the input wave, prior topassing the input wave through each of the at least two physical spatialmasks a number of times, or (2) moving the intensity sensor spatiallyafter passing the input wave through each of the at least two physicalspatial masks a number of times, to generate superresolved amplitude andphase information of the input wave.

P31. The method of P21, P22, P26, P27, P28, P29 or P30 furthercomprising performing a generalized Fourier transform on the inputthrough a lens prior to separately applying the input to each of the atleast two masks.

P32. The method of any of the preceding potential claims wherein eachmask includes an outer border that sets phase and amplitude of pointscoinciding with the outer border to zero.

P33. The method of any of P21 through P32, wherein the at least twomasks consists of one pair of complementary unipolar binary masks.

P34. The method of any of P21 through P32, wherein the pair of masksinclude unity elements that have a phase factor.

P35. The method of any of P21 through P32, wherein the at least twomasks consists of two pairs of complementary unipolar binary masks.

P36. The method of any of the preceding claims further comprising, aftercompletion, using the totagram to generate a representation of theinformation embedded in the reconstructed amplitude and phase of theinput.

P37. The method of any of the preceding claims, wherein iterativelyprocessing the plurality of complex phasorgrams comprises

-   -   (a) processing the plurality of complex phasorgrams to obtain a        single estimate of the input by performing on the complex        phasorgrams an inverse generalized Fourier transform and        averaging complex information at each point at corresponding        locations, and possibly some other additional optimization        steps;    -   (b) passing the single estimate of the input through a process        replicating each of the masks to obtain a plurality of        intermediate arrays;    -   (c) performing a generalized fast Fourier transform on each of        the intermediate arrays and replacing amplitude values at each        point in the transformed intermediate arrays with corresponding        recorded amplitude values, and possibly some other additional        optimization steps, to generate another plurality of complex        phasorgrams; and    -   (d) repeating step (a) for another plurality of complex        phasorgrams followed by steps (b) and (c) until convergence is        achieved, wherein upon completion the single estimate of the        input is the totagram.

P38. The method of P37, wherein convergence is determined by any of (1)when a squared difference between successive single estimates reach apredetermined threshold, and (2) when a given number of iterations ofstep (a) is completed.

P39. A system for recovering phase information from an input wavecomprising:

-   -   an optical lens system having an input with a spectral output;    -   at least two physical spatial masks, each of the at least two        physical spatial masks being disposed at the input of the        optical lens system for receiving the input wave;    -   wherein the at least two physical spatial masks are configured        to separately modify the input wave, and wherein the optical        lens system effects a generalized Fourier transform on the        separately modified waves to produce transformed waves;    -   at least one sensor system configured to record amplitude values        at an array of points of each transformed wave at the spectral        plane to produce phasorgrams; and    -   a digital processor configured to:        -   associate a phase value with each point on each phasorgram            to form a plurality of complex phasorgrams; and        -   iteratively process the plurality of complex phasorgrams            until convergence is achieved to produce a totagram            constituting a reconstructed input wave with amplitude and            phase information.

P40. The system of P39, further comprising a beam splitter configured toprovide the input wave to each of the at least two physical spatialmasks in parallel.

P41. The system of P39, further comprising a spatial light modulatorconfigured to implement the at least two physical spatial masksswitching from one of the masks to another of the at least two physicalspatial masks such that the input is individually received in sequenceby each of the at least two physical spatial masks.

P42. The system of P39, further comprising a micromirror arrayconfigured to implement the at least two physical spatial masksswitching from one of the masks to another of the at least two physicalspatial masks such that the input is individually received in sequenceby each of the at least two physical spatial masks.

P43. The system of any of P39 through P42, wherein the at least onesensor system is an intensity sensor.

P44. The system of P39 through P43, further comprising a second lenspositioned to receive the input wave en route to the at least twophysical spatial masks.

P45. The system of any of P39 through P44 wherein each mask includes anouter border that sets phase and amplitude of points coinciding with theouter border to zero.

P46. The system of any of the P39 through P45, wherein the at least twomasks include a unity mask.

P47. The system of P46 wherein the at least two masks consist of a unitymask and a complex phase mask.

P48. The system of P47, wherein the complex phase mask comprises abipolar binary mask.

P49. The system of any of potential claims P39 through P46, wherein theat least two masks consist of the unity mask and a pair of masks,wherein the masks in the pair are complementary with each other withrespect to amplitude.

P50. The system of P49, wherein the pair of masks are complementaryunipolar binary masks.

P51. The system of P49, wherein the pair of masks include unity elementsthat have a phase factor.

P52. The system of any of potential claims P39 through P45, wherein theat least two masks comprise at least one a pair of masks, wherein themasks in the pair are complementary with each other with respect toamplitude.

P53. The system of P52, wherein the pair of masks are complementaryunipolar binary masks.

P54. The system of P53, wherein the at least two masks consist of thepair of complementary unipolar binary masks.

P55. The system of P52, wherein the pair of masks include unity elementsthat have a phase factor.

P56. The system of any of P39 through P55, wherein to iterativelyprocess the plurality of complex phasorgrams comprises

-   -   (a) processing the plurality of complex phasorgrams to obtain a        single estimate of the input waveform by performing on the        complex phasorgrams an inverse generalized Fourier transform,        averaging complex information at each point at corresponding        locations, and possibly some other additional optimization        steps;    -   (b) passing the single estimate of the input through a process        replicating each of the masks to obtain a plurality of        intermediate arrays;    -   (c) performing a generalized fast Fourier transform on each of        the intermediate arrays and replacing amplitude values at each        point in the transformed intermediate arrays with corresponding        recorded amplitude values to generate another plurality of        complex phasorgrams, and possibly some other additional        optimization steps; and    -   (d) repeating step (a) for the another plurality of complex        phasorgrams followed by steps (b) and (c) until convergence is        achieved, wherein upon completion the single estimate of the        input is the totagram.

P57. The system of P56, wherein convergence is determined by any of (1)when a squared difference between successive single estimates reach apredetermined threshold, and (2) when a given number of iterations ofstep (a) is completed.

P58. The method of P15, wherein the pair of masks are complementarybipolar binary masks.

P59. The method of any of P21 through P32, wherein the at least twomasks consists of one pair of complementary bipolar binary masks.

P60. The system of P49, wherein the pair of masks are complementarybipolar binary masks.

The embodiments of the invention described above are intended to bemerely exemplary; numerous variations and modifications will be apparentto those skilled in the art. All such variations and modifications areintended to be within the scope of the present invention as defined inany appended claims.

What is claimed is:
 1. A method for recovering phase information from anarray of points, each point having an amplitude, the method comprising:providing at least one transformation unit having an input and aspectral output and four masks including two pairs of masks, wherein themasks in a pair are complementary with each other with respect toamplitude; separately applying the array of points at the input to eachof the four masks to generate a modified input from each of the masks;performing, by the at least one transformation unit, a generalizedFourier transform on each modified input to produce a transformedmodified input from each modified input; recording amplitude values atan array of points of each transformed modified input to producephasorgrams; associating a phase value with each point on eachphasorgram to form a plurality of complex phasorgrams; and iterativelyprocessing the plurality of complex phasorgrams, until convergence isachieved to produce a totagram constituting a reconstructed input withamplitude and phase information.
 2. The method of claim 1 wherein eachmask includes an outer border that sets phase and amplitude of pointscoinciding with the outer border to zero.
 3. The method of claim 1wherein each pair of masks includes complementary unipolar binary masks.4. The method of claim 1, wherein the input is a wave and the masks arephysical spatial masks and further comprising: switching from one of thephysical spatial masks to another of the physical spatial masks suchthat the input is individually received in sequence by each of thephysical spatial masks.
 5. The method of claim 1, wherein the input is awave and the masks are physical spatial masks and further comprising:splitting the input wave so that it is individually received in parallelby each of the physical spatial masks.
 6. The method of claim 1, whereinthe input comprises a wave and the masks comprise physical spatial masksimplemented in real time by optical devices including any of spatiallight modulators and micromirror arrays.
 7. The method of claim 6,further comprising performing (1) linear phase modulation on the inputwave, prior to passing the input wave through each of the physicalspatial masks a number of times, or (2) moving the intensity sensorspatially after passing the input wave through each of the physicalspatial masks a number of times, to generate superresolved amplitude andphase information of the input wave.
 8. The method of claim 6, whereinrecording is performed by an intensity sensor and the at least onetransformation unit comprises a lens system.
 9. The method of claim 1,further comprising, after completion, using the totagram to generate arepresentation of the information embedded in the reconstructedamplitude and phase of the input.
 10. A system for recovering phaseinformation from an input wave comprising: an optical lens system havingan input with a spectral output; at least two physical spatial masksincluding at least one pair of masks binary in amplitude, wherein themasks in a pair are complementary with each other with respect toamplitude, each of the at least two physical spatial masks beingdisposed at the input of the optical lens system for receiving the inputwave; wherein the at least two physical spatial masks are configured toseparately modify the input wave, and wherein the optical lens systemeffects a generalized Fourier transform on each of the separatelymodified waves to produce a transformed wave from each of the separatelymodified input waves; at least one sensor system configured to recordamplitude values at an array of points of each transformed wave at thespectral plane to produce a phasorgram from each transformed wave; and adigital processor configured to: associate a phase value with each pointon each phasorgram to form a plurality of complex phasorgrams; anditeratively process the plurality of complex phasorgrams untilconvergence is achieved to produce a totagram constituting areconstructed input wave with amplitude and phase information.
 11. Thesystem of claim 10, wherein each physical spatial mask is surrounded byan outer border that blocks the wave, thereby setting phase andamplitude of points that coincide with the border to zero.
 12. Thesystem of claim 10, wherein to iteratively process the plurality ofcomplex phasorgrams comprises (a) processing the plurality of complexphasorgrams to obtain a single estimate of the input wave by performingon the complex phasorgrams an inverse generalized Fourier transform andaveraging complex information at each point at corresponding locations;(b) passing the single estimate of the input wave through a processreplicating each of the physical spatial masks to obtain a plurality ofintermediate arrays; (c) performing a generalized fast Fourier transformon each of the intermediate arrays and replacing amplitude values ateach point in the transformed intermediate arrays with correspondingrecorded amplitude values to generate another plurality of complexphasorgrams; and (d) repeating step (a) for the another plurality ofcomplex phasorgrams followed by steps (b) and (c) until convergence isachieved, wherein upon completion the single estimate of the input isthe totagram.
 13. The system of claim 12, wherein convergence isdetermined by any of (1) when a difference between successive singleestimates reach a predetermined threshold, and (2) when a given numberof iterations of step (a) is completed.
 14. The system of claim 10,wherein the at least two physical spatial masks includes four masksincluding two pairs of masks, wherein the masks in a pair arecomplementary with each other with respect to amplitude.
 15. The systemof claim 10 wherein each pair of masks includes complementary unipolarbinary masks.
 16. A method for recovering phase information from anarray of points, each point having an amplitude, the method comprising:providing at least one transformation unit having an input and aspectral output and at least two masks including at least one pair ofmasks binary in amplitude, wherein the masks in a pair are complementarywith each other with respect to amplitude, each of the at least twomasks configured to operate upon the input of the at least onetransformation unit; separately applying the array of points at theinput to each of the at least two masks to generate a modified inputfrom each of the masks; performing, by the at least one transformationunit, a generalized Fourier transform on each modified input to producea transformed modified input from each modified input; recordingamplitude values at an array of points of each transformed modifiedinput to produce phasorgrams; associating a phase value with each pointon each phasorgram to form a plurality of complex phasorgrams; anditeratively processing the plurality of complex phasorgrams untilconvergence is achieved to produce a totagram constituting areconstructed input with amplitude and phase information, whereiniteratively processing the plurality of complex phasorgrams comprises:(a) processing the plurality of complex phasorgrams to obtain a singleestimate of the input by performing on the complex phasorgrams aninverse generalized Fourier transform and averaging complex informationat each point at corresponding locations; (b) passing the singleestimate of the input through a process replicating each of the masks toobtain a plurality of intermediate arrays; (c) performing a generalizedfast Fourier transform on each of the intermediate arrays and replacingamplitude values at each point in the transformed intermediate arrayswith corresponding recorded amplitude values, to generate anotherplurality of complex phasorgrams; and (d) repeating step (a) for anotherplurality of complex phasorgrams followed by steps (b) and (c) untilconvergence is achieved, wherein upon completion the single estimate ofthe input is the totagram.
 17. The method of 16 wherein the array ofpoints is a coherent light wave.
 18. The method of 16 wherein the arrayof points is digital data.
 19. The method of claim 16 wherein the atleast one transformation unit is a generalized Fourier transform processrun on a digital processor.
 20. The method of claim 16 wherein the atleast two masks are implemented on a digital processor.
 21. The methodof claim 16, wherein the at least one transformation unit is a lenssystem.
 22. The method of claim 16, wherein the masks are physicalspatial masks and further comprising switching from one of the at leasttwo physical spatial masks to another of the at least two physicalspatial masks such that the input is individually received in sequenceby each of the at least two physical spatial masks.
 23. The method ofclaim 16, wherein the masks are physical spatial masks and furthercomprising splitting the input wave so that it is individually receivedin parallel by each of the at least two physical spatial masks.
 24. Themethod of claim 16, wherein the at least two masks comprise physicalspatial masks implemented in real time by optical devices including anyof spatial light modulators and micromirror arrays.
 25. The method ofclaim 16, wherein recording is performed by an intensity sensor.
 26. Themethod of claim 16, further comprising (1) performing linear phasemodulation on the input wave, prior to passing the input wave througheach of the at least two physical spatial masks a number of times, or(2) moving the intensity sensor spatially after passing the input wavethrough each of the at least two physical spatial masks a number oftimes, to generate superresolved amplitude and phase information of theinput wave.
 27. The method of claim 16 wherein each mask includes anouter border that sets phase and amplitude of points coinciding with theouter border to zero.
 28. The method of claim 16, wherein the at leasttwo masks consists of one pair of complementary unipolar binary masks.29. The method of claim 16, wherein the pair of masks include unityelements that have a phase factor.
 30. The method of claim 16, whereinthe at least two masks consists of two pairs of complementary unipolarbinary masks.
 31. The method of claim 16, wherein convergence isdetermined by any of (1) when a squared difference between successivesingle estimates reach a predetermined threshold, and (2) when a givennumber of iterations of step (a) is completed.